
United States Patent and Trademark Office 



UNITED STATES DEPARTMENT OF COMMERCE 
United States Patent and Trademark OfTice 
Address: COMMISSIONER FOR PATENTS 
P.O. Box 1450 

Alexandria, Virginia 223 1 3- 1 450 
www.uspto.gov 



I ATTORNEY DOCKET NO. | CONFIRMATION NO~ 



APPLICATION NO. 



FILING DATE 



FIRST NAMED INVENTOR 



09/749,165 



12/27/2000 



7590 01/25/2006 

HEWLETT-PACKARD COMPANY 
Intellectual Property Administration 
P. O. Box 272400 
Fort Collins, CO 80527-2400 



Robert E. Sobol 



10003840-1 



4943 



EXAMINER 



TUCKER, WESLEY J 



ART UNIT 



PAPER NUMBER 



2623 

DATE MAILED: 01/25/2006 



Please find below and/or attached an Office communication concerning this application or proceeding. 



PTO-90C (Rev. 10/03) 



Office Action Summary 


Application No. 

09/749,165 


Applicant(s) 

SOBOL, ROBERT E. 


Examiner 

Wes Tucker 


Art Unit 

2623 





- The MAILING DATE of this communication appears on the cover sheet with the correspondence address 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS, 
WHICHEVER IS LONGER. FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If NO period for reply is specified above, the maximum statutory period vaII apply and vAW expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply writhin the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1)K Responsive to communication(s) filed on 04 November 2005 . 
2a)M This action is FINAL. 2b)n This action is non-final. 

3) 0 Since this application is in condition for allowance except for fomnal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1 935 CD. 1 1 , 453 O.G. 21 3. 

Disposition of Claims 

4) IE Claim(s) 1-18,20.21 arid 23-32 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) 0 Claim(s) is/are allowed. 

6) IE Clalm(s) 1-18.20.21 and 23-32 is/are rejected. 
?)□ Claim(s) is/are objected to. 

8) n Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) 0 The specification is objected to by the Examiner. 

10) ^ The drawing(s) filed on 27 December 2000 is/are: a)^ accepted or b)n objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing{s) is objected to. See 37 CFR 1.121(d). 

11) 0 The oath or declaration is objected to by the Examiner. Note the attached Office Action orforni PTO-152. 

Priority under 35 U.S.C. § 119 

12) 0 Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 
a)n All b)n Some * 0)0 None of: 

1 .□ Certified copies of the priority documents have been received. 

2. n Certified copies of the priority documents have been received in Application No. . 

3. n Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 



Attachment(s) 

1) □ Notice of References Cited (PTO-892) 

2) □ Notice of Draftsperson's Patent Drawing Review (PTO-948) 
J^3) ^ Information Disclosure Statement(s) (PTO-1449 or PTO/SB/08) 

Paper No(s)/Mail Date^^gR //" H^OS 



4) n Interview Summary (PTO-413) 

Paper No(s)/Mail Date. . 

5) O Notice of Informal Patent Application (PTO-152) 

6) □ Other: . 



U.S. Patent and Trademark Office 
PTOL-326 (Rev, 7-05) 



Offlce Action Summary 



Part of Paper No./Mail Date 14 



Application/Control Number: 09/749,165 Page 2 

Art Unit: 2623 

DETAILED ACTION 



Response to Arguments and Amendments 

1 . Applicant's response to the Office Action, filed Nov. 4"^ 2005, has been 
entered and made of record. 

2. Applicant has amended claims 1, 2, 9-10, 12, 20-21 and 28-30. Claims 1- 
18, 20-21 and 23-32 are pending. 

3. Applicant's arguments have been fully considered and are not persuasive 
for at least the following reasons: 

4. Applicants arguments repeatedly presented have focused on the claim 
that the reference to Hillebrand does not disclose the claimed feature of: 

wherein said image enfiancer is configured to initiate, without user 
intervention, manipulation of said portion for enhancing said appearance in 
response to location of said facial blemish. 

Examiner has pointed and will point again to the multiple passages in Hillebrand 
where it is disclosed that steps are performed automatically by a controller 200 (column 
6, lines 15-16, column 7, lines 17-21 and 65-66, column 8, lines 33-36, 46-51 and 63- 
65, column 9, lines 5-7, 15-17 and 26-28, column 10, lines 51-52, column 12, lines 2-3 
and 10-14). Hillebrand consistently states, "in one embodiment the steps are performed 
by the controller 200." This declaration is interpreted as the steps in the process take 
place entirely without user input based on data already programmed into the image 
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processor or controller. Although Hillebrand allows for user input at nearly every step of 
the process, it is also indicated clearly that in one embodiment, the steps are performed 
by the controller without user input. 

Applicant argued repeatedly that Hillebrand teaches away from the present 
invention because Hillebrand allows for user input at several stages of the process. 

For the sake of the argument Examiner cited the reference to Lawton that 
teaches automatically identifying and removing spots such as facial blemishes 
automatically (column 4, lines 18-25). 

Applicant now argues in view of the reference to Lawton that no proper 
motivation has been given and even that there is no reason or motivation that one 
would want to combine the references, that improper hindsight has been used to 
reconstruct the elements of the claim, and that the reference of Hillebrand teaches away 
from performing the step: 

wherein said image enhancer is configured to initiate, without user 

intervention, manipulation of said portion for enhancing said appearance in 

response to location of said facial blemish. 

Examiner will now explain why the combination of the cited art to Lawton and 
Hillebrand is analogous and motivated, why hindsight is not required, and why 
Hillebrand does not teach away from performing the step automatically. 

Hillebrand and Lawton are analogous art in that they bothjeach seeking to 
remove facial blemishes, a very specific function in image processing. The motivation 
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to combine the Hiiiebrand and Lawton references is the same motivation to use a 
computer to automate an image processing function and indeed any process automated 
by a computer. When processes are automated and do not require user intervention, its 
just one less thing the user or customer or technician has to do to achieve the desired 
result of the computer program. 

In response to applicant's argument that the examiner's conclusion of 
obviousness is based upon improper hindsight reasoning, it must be recognized that 
any judgment on obviousness is in a sense necessarily a reconstruction based upon 
hindsight reasoning. But so long as it takes into account only knowledge which was 
within the level of ordinary skill at the time the claimed invention was made, and does 
not include knowledge gleaned only from the applicant's disclosure, such a 
reconstruction is proper. See In re McLaughlin, 443 F.2d 1392, 170 USPQ 209 (CCPA 
1971 ). It must be clear that absolutely no hindsight is required to see the benefit of 
modifying the reference of Hiiiebrand to automate what is essentially one step in the 
process of the facial image enhancement. This is not an unreasonable construction, it 
is a combination of analogous art to teach that automation of the claimed step would be 
an obviously motivated modification to the reference of Hiiiebrand and indeed an image 
processor for the same reasons that any process is automated or performed by a 
processor and requires no user dictation for the process to perform its given functions. 
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Any number of motivations can be given for automating the questioned step: less 
room for user error, faster processing, allows user the freedom to do something else, 
etc. It is exceedingly well known in the art to automate processes with computers. In 
the particular case of Lawton, it is taught that "attributes for the identified spot are then 
used to identify similar spots throughout the digital image, which are also automatically 
removed." This is a clear example of how once the computer learns what to do, it 
continues to do it unaided. This is clearly beneficial in Hillebrand and analogous to the 
nature in which Hillebrand teaches automation by the controller at any number of steps. 
It is more than reasonable to assume that Hillebrand would benefit from automation of 
the step. 

Applicant further argues that Hillebrand teaches away from the automation of the 
step in question because, as inferred by Applicant on page 19 of the arguments it is 
stated that: 

Hillebrand specifically teaches that "Simulating improvements may 
be useful when the operator is recommending treatment of when the 
operator is recommending a treatment using a product which eliminates 
and/or hides skin defects to show the analyzed person the potential 
benefits of the product(s)." Allowing a user to submit inputs for controlling 
not only the magnitude of the improvements, as described above, but also 
for controlling the timing of the alleged image enhancements appears to be 
desirable in achieving the stated purpose of Hillebrand, and obviousness is 
concerned not with what is feasible but rather with what is desirable." 
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It appears that Applicant is now inferring what allegedly may or may not be 
desirable in the operation of the invention of Hillebrand as justification for teaching away 
from the present claims. 

Even if Hillebrand was concerned with the timing of the image enhancements, it 
would still be obvious to perform the enhancements automatically without user 
intervention as claimed perhaps on say a timer. How much time would Hillebrand want 
between the time of automatically detecting a defect and automatically enhancing it? It 
is not a question of whether it is feasible or whether it is desirable. Performing the 
image enhancement automatically is an obvious modification for Hillebrand alone and is 
clearly an obvious and motivated adjustment in view of the Lawton reference. 

From the discussion it must now be clear that the combination of the references 
to Kinjo Hillebrand and Lawton are obvious and motivated and desirable for the reasons 
stated above. 

Applicant further argues the use of the Kinjo reference to disclose detecting 
multiple faces with regard to claims 23-29. Applicant insists that Kinjo only detects 
multiple face candidate areas. It should be noted that the purpose of the invention of 
Kinjo is to detect face regions using overlapping detected regions and determining face 
regions based on variable thresholds and conditions for facial regions. It should be 
understood that if there are multiple face regions in a given image, that the invention of 
Kinjo will find them all depending on the various conditions and thresholds set for facial 
region determination (column 19, lines 1-20, column 20, lines 40-55 and column 21, 
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lines 45-65). Kinjo also teaches performing enhancement on the extracted face regions 
once they are detected. It should now be understood that Kinjo discloses detecting all 
facial candidate regions in an image and based on the conditions determines all existing 
facial regions. 

Applicant also argues that there is no obvious motivation as to why Kinjo would 
detect multiple faces and enhance all of them. The reason for enhancing each face 
region found requires little motivation or case for obviousness. The motivation for 
performing enhancement on multiple faces is the exact same motivation for performing 
it on one detected face, to enhance image quality or appearance, plain and simple. 

All of Applicants arguments with regard to the remaining dependent claims rely 
on the discussions regarding the combination of the three references with regard to 
claim 1 and are therefore accordingly addressed in the explanation above. 

The previously presented rejection is therefore maintained and accordingly made 

FINAL. 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth 
in section 1 02 of this title, if the differences between the subject matter sought to be patented and the 
prior art are such that the subject matter as a whole would have been obvious at the time the invention 
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was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability 
shall not be negatived by the manner in which the invention was made. 

5. Claims 1-3, 5-12, and 14-21, 23-32 are rejected under 35 U.S.C. 103(a) 
as being unpatentable over the combination of U.S. Patent 6,445,819 to Kinjo and U.S. 
Patent 6,571,003 to Hillebrand et al. and further in view of U.S. Patent 6,160,923 to 
Lawton et al. 

With regard to claim 1 , Kinjo discloses an image enhancement system, 
comprising memory for storing digital data that defines a graphical image (Fig.1 , 
element 48) and a face detector configured to analyze said digital data and to 
automatically identify facia! data within said digital data stored in said memory (Fig.2, 
element 100). 

Kinjo discloses an image enhancer, but does not disclose automatically locating 
a facial blemish defined by a portion of said facial data based on a proximity of said 
facial blemish relative to said facial feature within said graphical image said image 
enhancer further configured to automatically compensate for said facial blemish by 
automatically manipulating said portion such that an appearance of said facial blemish 
is enhanced within said graphical image, wherein said image enhancer is configured 
to initiate, without user intervention, manipulation of said portion for enhancing 
said appearance in response to location of said facial blemish by said image 
enhancer. 
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Hillebrand discloses an image enhancer configured to automatically detect and 
locate defect areas in a face image and to create an improved or worsened area for 
each of the defect areas (column 2, lines 15-30). Once a face has been detected in a 
digital image, several different processes are used to change portions of that image in 
order to enhance the appearance of that image. Therefore it would have been obvious 
to one of ordinary skill in the art at the time of invention to employ Hillebrand's facial 
enhancement system to modify all or part of the facial image detected in the face 
detection of Kinjo in order to enhance the appearance of that image. 

With regard to the argued part of the claim in italics above, Hillebrand also 
discloses wherein the image enhancement occurs without any user intervention (column 
6, lines 15-16, column 7, lines 17-21 and 65-66, column 8, lines 33-36, 46-51 and 63- 
65, column 9, lines 5-7, 15-17 and 26-28, column 10, lines 51-52, column 12, lines 2-3 
and 10-14). Hillebrand consistently states, "in one embodiment the steps are performed 
by the controller 200." This declaration is interpreted as the steps in the process take 
place entirely without user input based on data already programmed into the image 
processor or controller. Although Hillebrand allows for user input at nearly every step of 
the process, it is also indicated clearly that in one embodiment, the steps are performed 
by the controller. 

For the sake of argument, if it cannot be agreed as to whether or not Hillebrand 
allows for user intervention before the manipulation of said portion for enhancing said 
appearance in response to identification of said facial blemish by said image enhancer. 
Examiner points to the reference of U.S. Patent 6,160,923 to Lawton et al. Lawton 
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discloses a device to remove spots or anomalies such as facial blemishes (column 3, 
lines 3-6) wherein once the area of the blemish is identified, the enhancement of the 
area of the blemish is performed automatically (column 3, lines 13-17) without user 
intervention. Lawton further teaches that attributes of identified blemishes to be 
removed are gathered and using those attributes such imperfections can be 
automatically removed throughout the image without user intervention (column 4, lines 
18-25). 

Hillebrand and Lav\rton are analogous art in that they both teach seeking to 
remove facial blemishes, a very specific function in image processing. The motivation 
to combine the Hillebrand and Lawton references is the same motivation to use a 
computer to automate an image processing function and indeed any process automated 
by a computer. When processes are automated and do not require user intervention, its 
just one less thing the user or customer or technician has to do to achieve the desired 
result of the computer program. Furthermore no hindsight is required to see the benefit 
of modifying the reference of Hillebrand to automate what is essentially ope step in the 
process of the facial image enhancement. Any number of motivations can be given for 
automating this step: less room for user error, faster processing, allows user the 
freedom to do something else, etc. It is exceedingly well known in the art to automate 
processes with computers. In the particular case of Lawton, it is taught that "attributes 
for the identified spot are then used to identify similar spots throughout the digital image, 
which are also automatically removed." This is a clear example of how once the 
computer learns what to do, it continues to do it unaided. Therefore it would have been 
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obvious to one of ordinary skill in the art at the time of invention to enable automatic 
image enhancement without user intervention once a blemish has been identified as 
taught by Lawton to enable automated processing of images, and all of the mentioned 
benefits of automating computer processing listed above in conjunction with the 
controller operated method of Hillebrand already disclosed. 

With regard to claim 2, Kinjo and Hillebrand disclose input devices configured to 
receive an input. Kinjo discloses a scanner (Fig.1, element 12). Hillebrand discloses 
user input indicative of a facial blemish tvpe (Fig. 9, step 902) and wherein said location 
of said facial blemish by is based on said input (Fig. 9, element 904). 

Hillebrand allows for inputs and specifications obtained from a user interface on 
what operations to perform and how to determine facial portions to analyze and 
enhance (column 5, lines 30-42 and column 6, lines 29-42). In the combination of Kinjo 
and Hillebrand, Hillebrand's image enhancer would operate in accordance with the input 
configurations to perform operations on the image input received by Kinjo's input 
device. The portion of the image to be modified or the facial feature selected is based 
on these inputs. 

With regard to claim 3, Kinjo discloses said image enhancer to process color 
correction, but does not specifically refer to color blending. Hillebrand discloses said 
image enhancer to manipulate portions of an image by blending color values associated 
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with the defective portion with the color of the skin surrounding the defective portion 
(column 2, lines 25-30 and column 12, lines 13-18). 

With regard to claim 5, Kinjo discloses said image enhancer, wherein 
manipulating said portion, sharpens said appearance of said facial feature (column 10, 
lines 65-67). 

With regard to claim 6, Kinjo discloses said image enhancer, wherein 
manipulating said portion, changes a color of said facial feature (column 10, lines 60- 
65). 

With regard to claim 7, Kinjo discloses said system including an image capturing 
device configured to receive an image of a scene and to produce said digital data based 
on said image received by said image capturing device (Fig.1, element 12). In this 
embodiment the image capture device is a scanner. 

With regard to claim 8, Kinjo discloses said image capturing device including a 
lens for receiving said image and an image converter for producing said digital data 
based on said image (Fig. 1, elements 12, 28, and 32). The scanner contains a CCD 
with lens (28) and analog to digital converter (32). 
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With regard to claim 9, Kinjo discloses an automatic image enhancement system, 
comprising means for storing digital data that defines a graphical image (Fig.1, element 
48) and face detecting means for analyzing said digital data and for automatically 
identifying facial data within said digital data stored in said storing means (Fig. 2, 
element 100). 

Kinjo discloses an image enhancing means, but does not disclose means for 
searching said identified facial data for a particular facial feature and for automatically 
locating a facial blemish defined by a portion of said facial data based on a proximity of 
said facial blemish relative to said facial feature within said graphical image said image 
enhancer further configured to automatically compensate for said facial blemish by 
automatically manipulating upon locating said facial blemish, said portion such that an 
appearance of said facial feature is enhanced within said graphical image, wherein 
said image enfiancer is configured to initiate, wittiout user intervention, 
manipulation of said portion for enhancing said appearance in response to 
identification of said facial blemish by said image enhancer. 

Hillebrand discloses an image enhancer configured to automatically detect and 
locate defect areas in a face image and to create an improved or worsened area for 
each of the defect areas (column 2, lines 15-30). Once a face has been detected in a 
digital image, several different processes are used to change portions of that image in 
order to enhance the appearance of that image. Therefore it would have been obvious 
to one of ordinary skill in the art at the time of invention to employ Hillebrand's facial 
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enhancement system to modify all or part of the facial image detected in the face 
detection of Kinjo in order to enhance the appearance of that image. 

With regard to the amended part of the claim underlined above, Hillebrand also 
discloses wherein the image enhancement occurs without any user intervention (column 
6, lines 15-16, column 7, lines 17-21 and 65-66, column 8, lines 33-36, 46-51 and 63- 
65, column 9, lines 5-7, 15-17 and 26-28, column 10, lines 51-52, column 12, lines 2-3 
and 10-14). Hillebrand consistently states, "in one embodiment the steps are performed 
by the controller 200." This declaration is interpreted as the steps in the process take 
place entirely without user input based on data already programmed into the image 
processor or controller. Although Hillebrand allows for user input at nearly every step of 
the process, it is also indicated clearly that in one embodiment, the steps are performed 
by the controller. 

For the sake of argument, if it cannot be agreed as to whether or not Hillebrand 
allows for user intervention before the manipulation of said portion for enhancing said 
appearance in response to identification of said facial blemish by said image enhancer, 
Examiner points to the reference of U.S. Patent 6,160,923 to Lawton et al. Lawton 
discloses a device to remove spots or anomalies such as facial blemishes (column 3, 
lines 3-6) wherein once the area of the blemish is identified, the enhancement of the 
area of the blemish is performed automatically (column 3, lines 13-17) without user 
intervention. Lawton further teaches that attributes of identified blemishes to be 
removed are gathered and using those attributes such imperfections can be 
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automatically removed throughout the image without user intervention (column 4, lines 
18-25). 

Hillebrand and Lawton are analogous art in that they both teach seeking to 
remove facial blemishes, a very specific function in image processing. The motivation 
to combine the Hillebrand and Lawton references is the same motivation to use a 
computer to automate an image processing function and indeed any process automated 
by a computer. When processes are automated and do not require user intervention, its 
just one less thing the user or customer or technician has to do to achieve the desired 
result of the computer program. Furthermore no hindsight is required to see the benefit 
of modifying the reference of Hillebrand to automate what is essentially one step in the 
process of the facial image enhancement. Any number of motivations can be given for 
automating this step: less room for user error, faster processing, allows user the 
freedom to do something else, etc. It is exceedingly well known in the art to automate 
processes with computers. In the particular case of Lawton, it is taught that "attributes 
for the identified spot are then used to identify similar spots throughout the digital image, 
which are also automatically removed." This is a clear example of how once the 
computer learns what to do, it continues to do it unaided. Therefore it would have been 
obvious to one of ordinary skill in the art at the time of invention to enable automatic 
image enhancement without user intervention once a blemish has been identified as 
taught by Lawton to enable automated processing of images, and all of the mentioned 
benefits of automating computer processing listed above in conjunction with the 
controller operated method of Hillebrand already disclosed. 
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With regard to claim 10, Kinjo discloses a method for enhancing graphical 
images, comprising: 

receiving digital data defining a graphical image (Fig. 1, elements 32, 36 and 14); 

automatically detecting facial data within said digital data (Fig. 2, element 100). 

Kinjo does not explicitly disclose searching said facial data for data that defines a 
particular facial feature and automatically locating, a facial blemish defined by a set of 
said digital data based on a proximity of said facial blemish relative to said particular 
facial feature within said graphical image; and automatically compensating for said 
facial blemish in response to said locating , without user intervention, said compensating 
comprising manipulating said set of digital data. 

Hillebrand discloses an image enhancer configured to automatically detect and 
locate defect areas in a face image and to create an improved or worsened area for 
each of the defect areas (column 2, lines 15-30). Once a face has been detected in a 
digital image, several different processes are used to change portions of that image in 
order to enhance the appearance of that image. Therefore it would have been obvious 
to one of ordinary skill in the art at the time of invention to employ Hillebrand's facial 
enhancement system to modify all or part of the facial image detected in the face 
detection of Kinjo in order to enhance the appearance of that image. 

With regard to the amended part of the claim underlined above, Hillebrand also 
discloses wherein the image enhancement occurs without any user intervention (column 
6, lines 15-16, column 7, lines 17-21 and 65-66, column 8, lines 33-36, 46-51 and 63- 
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65, column 9, lines 5-7, 15-17 and 26-28, column 10, lines 51-52, column 12, lines 2-3 
and 10-14). Hillebrand consistently states, "in one embodiment the steps are performed 
by the controller 200." This declaration is interpreted as the steps in the process take 
place entirely without user input based on data already programmed into the image 
processor or controller. Although Hillebrand allows for user input at nearly every step of 
the process, it is also indicated clearly that in one embodiment, . the steps are performed 
by the controller. 

For the sake of argument, if it cannot be agreed as to whether or not Hillebrand 
allows for user intervention before the manipulation of said portion for enhancing said 
appearance in response to identification of said facial blemish by said image enhancer, 
Examiner points to the reference of U.S. Patent 6,160,923 to Lawton et al. Lawton 
discloses a device to remove spots or anomalies such as facial blemishes (column 3, 
lines 3-6) wherein once the area of the blemish is identified, the enhancement of the 
area of the blemish is performed automatically (column 3, lines 13-17) without user 
intervention. Lawton further teaches that attributes of identified blemishes to be 
removed are gathered and using those attributes such imperfections can be 
automatically removed throughout the image without user intervention (column 4, lines 
18-25). 

Hillebrand and Lawton are analogous art in that they both teach seeking to 
remove facial blemishes, a very specific function in image processing. The motivation 
to combine the Hillebrand and Lawton references is the same motivation to use a 
computer to automate an image processing function and indeed any process automated 
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by a computer. When processes are automated and do not require user intervention, its 
just one less thing the user or customer or technician has to do to achieve the desired 
result of the computer program. Furthermore no hindsight is required to see the benefit 
of modifying the reference of Hillebrand to automate what is essentially one step in the 
process of the facial image enhancement. Any number of motivations can be given for 
automating this step: less room for user error, faster processing, allows user the 
freedom to do something else, etc. It is exceedingly well known in the art to automate 
processes with computers. In the particular case of Lawton, it is taught that "attributes 
for the identified spot are then used to identify similar spots throughout the digital image, 
which are also automatically removed." This is a clear example of how once the 
computer learns what to do, it continues to do it unaided. Therefore it would have been 
obvious to one of ordinary skill in the art at the time of invention to enable automatic 
image enhancement without user intervention once a blemish has been identified as 
taught by Lawton to enable automated processing of images, and all of the mentioned 
benefits of automating computer processing listed above in conjunction with the 
controller operated method of Hillebrand already disclosed. 

With regard to Claims 11-17, they repeat the elements in claims 2-8 with 
reference to a method. With regard to depending claims 11-17, refer to the discussion 
of claims 2-8 respectively. Claims 11-17 are amended only by eliminating the word 
"step" in each claim. 
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With regard to claim 18, the combination of Kinjo and Hillebrand applies as 
discussed in regard to claim 9. Claim 9 cites all of the elements of claim 18, with the 
exception of explicitly naming a facial skin blemish. Hillebrand discloses this feature 
(column 2, lines 17-22 and column 6, lines 1-5). With regard to the new limitation 
wherein said image enhancer is further configured to automatically manipulate, upon 
locating said portion for enhancing an appearance of said skin blemish within said 
graphical image, Hillebrand also discloses wherein the image enhancement occurs 
without any user intervention (column 6, lines 15-16, column 7, lines 17-21 and 65-66, 
column 8, lines 33-36, 46-51 and 63-65, column 9, lines 5-7, 15-17 and 26-28, column 
10, lines 51-52, column 12, lines 2-3 and 10-14). Hillebrand consistently states, "in one 
embodiment the steps are performed by the controller 200." This declaration is 
interpreted as the steps in the process take place entirely without user input based on 
data already programmed into the image processor or controller. Although Hillebrand 
allows for user input at nearty every step of the process, it is also indicated cleariy that 
in one embodiment, the steps are performed by the controller. 

With regard to claim 19, Hillebrand discloses the image enhancer configured to 
locate at least one additional facial feature and locate said portion of said facial data 
defining said skin blemish by determining the likely proximity of said skin blemish to said 
located at least one additional facial feature. Hillebrand discloses several methods of 
sub-image determination (column 7, lines 16-40 and 60-65 and column 8, lines 5-10). 
Hillebrand discloses locating sub-images such as an under eye border and then 
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discloses enhancing the image portion according to the defect type in that area such as 
wrinkles blemishes red spots etc. 

With regard to claim 20, Hillebrand discloses the said blemish to be a wrinkle 
(column 8, lines 5-8). 

With regard to claim 21 , the discussions of claims 1 , 9 and 1 8 apply. The 
method is considered to be included in the system. 

With regard to claim 23, the combination of Kinjo and Hillebrand discloses the 
system of claim 1 , and Kinjo discloses detecting multiple face candidate regions (Fig. 2, 
element 102 and Fig. 3) and in the combination of Kinjo and Hillebrand, image 
enhancement would take place on any number of faces once they are detected. 

It should be noted that the purpose of the invention of Kinjo is to detect face 
regions using overlapping detected regions and determining face regions based on 
variable thresholds and conditions for facial regions. It should be understood that if 
there are multiple face regions in a given image, that the invention of Kinjo will find them 
all depending on the various conditions and thresholds set for facial region 
determination (column 19, lines 1-20, column 20, lines 40-55 and column 21, lines 45- 
65). Kinjo also teaches performing enhancement on the extracted face regions once 
they are detected. It should now be understood that Kinjo discloses detecting all facial 
candidate regions in an image and based on the conditions determines all existing facial 
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regions. Therefore it would have been obvious to one of ordinary skill in the art at the 
time of invention to enhance any number of face regions detected by the facial region 
detection system of Kinjo in combination by the face enhancements techniques taught 
by Hillebrand and Lawton in order to enhance all of the facial regions in an image. 

Hillebrand discloses wherein the image enhancement occurs without any user 
intervention (column 6, lines 15, 16, column 7, lines 17-21 and 65-66, column 8, lines 
33-36, 46-51 and 63-65, column 9. lines 5-7, 15-17 and 26-28, column 10, lines 51-52, 
column 12, lines 2-3 and 10-14). Hillebrand consistently states, "in one embodiment the 
steps are performed by the controller 200." This declaration is interpreted as the steps 
in the process take place entirely without user input based on data already programmed 
into the image processor or controller. Although Hillebrand allows for user input at 
nearly every step of the process, it is also indicated clearly that in one embodiment, the 
steps are performed by the controller. 

For the sake of argument, if it cannot be agreed as to whether or not Hillebrand 
allows for user intervention before the manipulation of said portion for enhancing said 
appearance in response to identification of said facial blemish by said image enhancer. 
Examiner points to the reference of U.S. Patent 6,160,923 to Lawton et al. Lawton 
discloses a device to remove spots or anomalies such as facial blemishes (column 3, 
lines 3-6) wherein once the area of the blemish is identified, the enhancement of the 
area of the blemish is performed automatically (column 3, lines 13-17) without user 
intervention. Lawton further teaches that attributes of identified blemishes to be 
removed are gathered and using those attributes such imperfections can be 
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automatically removed throughout the image without user intervention (column 4, lines 
18-25). 

Hillebrand and Lawton are analogous art in that they both teach seeking to 
remove facial blemishes, a very specific function in image processing. The motivation 
to combine the Hillebrand and Lawton references is the same motivation to use a 
computer to automate an image processing function and indeed any process automated 
by a computer. When processes are automated and do not require user intervention, its 
just one less thing the user or customer or technician has to do to achieve the desired 
result of the computer program. Furthermore no hindsight is required to see the benefit 
of modifying the reference of Hillebrand to automate what is essentially one step in the 
process of the facial image enhancement. Any number of motivations can be given for 
automating this step: less room for user error, faster processing, allows user the 
freedom to do something else, etc. It is exceedingly well known in the art to automate 
processes with computers. In the particular case of Lawton, it is taught that "attributes 
for the identified spot are then used to identify similar spots throughout the digital image, 
which are also automatically removed." This is a clear example of how once the 
computer learns what to do, it continues to do it unaided. Therefore it would have been 
obvious to one of ordinary skill in the art at the time of invention to enable automatic 
image enhancement without user intervention once a blemish has been identified as 
taught by Lawton to enable automated processing of images, and all of the mentioned 
benefits of automating computer processing listed above in conjunction with the 
controller operated method of Hillebrand already disclosed. 
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With regard to claim 24, the discussion of claim 23 applies. 

With regard to claim 25, the discussion of claim 23 applies. 

With regard to claim 26, the discussion of claim 23 applies. 

With regard to claim 27, the discussion of claim 23 applies. 

With regard to claim 28 Kinjo discloses an automatic image enhancement 
system, comprising memory (Fig. 1 , element 48) for storing digital data that defines a 
graphical image, said graphical image containing a plurality effaces (Fig, 2, elements 
100 and 102). Kinjo disclose multiple face candidate regions. It should be noted that 
the purpose of the invention of Kinjo is to detect face regions using overlapping 
detected regions and determining face regions based on variable thresholds and 
conditions for facial regions. It should be understood that if there are multiple face 
regions in a given image, that the invention of Kinjo will find them all depending on the 
various conditions and thresholds set for facial region determination (column 19, lines 1- 
20, column 20, lines 40-55 and column 21 , lines 45-65). Kinjo also teaches performing 
enhancement on the extracted face regions once they are detected, it should now be 
understood that Kinjo discloses detecting all facial candidate regions in an image and 
based on the conditions determines all existing facial regions. Therefore it would have 
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been obvious to one of ordinary skill in the art at the time of invention to enhance any 
number of face regions detected by the facial region detection system of Kinjo in 
combination by the face enhancements techniques taught by Hillebrand and Lawton in 
order to enhance all of the facial regions in an image. 

Kinjo also discloses a face detector configured to detect each of said faces (Fig. 
2, element 100). 

Kinjo discloses enhancing said faces once found (column 20, lines 40-55) but 
does not disclose an image enhancer configured to analyze said faces, said image 
enhancer further configured to automatically detect and enhance at least one respective 
facial blemish in each of said faces. 

Hillebrand discloses an image enhancer configured to automatically detect and 
locate defect areas in a face image and to create an improved or worsened area for 
each of the defect areas (column 2, lines 15-30). Once a face has been detected in a 
digital image, several different processes are used to change portions of that image in 
order to enhance the appearance of that image. This of course applies to multiple 
detected facial images. Therefore it would have been obvious to one of ordinary skill in 
the art at the time of invention to employ Hillebrand's facial enhancement system to 
modify all or part of the facial image detected in the face detection of Kinjo in order to 
enhance the appearance of that image. 

Hillebrand also discloses wherein the image enhancement occurs without any 
user intervention (column 6, lines 15-16, column 7, lines 17-21 and 65-66, column 8, 
lines 33-36, 46-51 and 63-65, column 9, lines 5-7, 15-17 and 26-28, column 10, lines 
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51-52, column 12, lines 2-3 and 10-14). Hillebrand consistently states, "in one 
embodiment the steps are performed by the controller 200." This declaration is 
interpreted as the steps in the process take place entirely without user input based on 
data already programmed into the image processor or controller. Although Hillebrand 
allows for user input at nearly every step of the process, it is also indicated clearly that 
in one embodiment, the steps are performed by the controller. 

Hillebrand further discloses w herein the image enhancer is configured to detect 
each said facial blemish by searching a respective one of said detected faces for a 
particular facial feature and determining the likely proximity of the facial blemish relative 
to the particular facial feature (column 2, lines 1-25). 

For the sake of argument, if it cannot be agreed as to whether or not Hillebrand 
allows for user intervention before the manipulation of said portion for enhancing said 
appearance in response to identification of said facial blemish by said image enhancer, 
Examiner points to the reference of U.S. Patent 6,160,923 to Lawton et al. Lawton 
discloses a device to remove spots or anomalies such as facial blemishes (column 3, 
lines 3-6) wherein once the area of the blemish is identified, the enhancement of the 
area of the blemish is performed automatically (column 3, lines 13-17) without user 
intervention. Lawton further teaches that attributes of identified blemishes to be 
removed are gathered and using those attributes such imperfections can be 
automatically removed throughout the image without user intervention (column 4, lines 
18-25). 
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Hillebrand and Lawton are analogous art in that they both teach seeking to 
remove facial blemishes, a very specific function in image processing. The motivation 
to combine the Hillebrand and Lawton references is the same motivation to use a 
computer to automate an image processing function and indeed any process automated 
by a computer. When processes are automated and do not require user intervention, its 
just one less thing the user or customer or technician has to do to achieve the desired 
result of the computer program. Furthermore no hindsight is required to see the benefit 
of modifying the reference of Hillebrand to automate what is essentially one step in the 
process of the facial image enhancement. Any number of motivations can be given for 
automating this step: less room for user error, faster processing, allows user the 
freedom to do something else, etc. It is exceedingly well known in the art to automate 
processes with computers. In the particular case of Lawton, it is taught that "attributes 
for the identified spot are then used to identify similar spots throughout the digital image, 
which are also automatically removed." This is a clear example of how once the 
computer learns what to do, it continues to do it unaided. Therefore it would have been 
obvious to one of ordinary skill in the art at the time of invention to enable automatic 
image enhancement without user intervention once a blemish has been identified as 
taught by Lawton to enable automated processing of images, and all of the mentioned 
benefits of automating computer processing listed above in conjunction with the 
controller operated method of Hillebrand already disclosed. 
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With regard to claim 29, the discussion of claim 28 applies. The method is 
disclosed in the use of the system. 

With regard to claim 30, the discussions of claims 28 and 29 apply. The 
discussion of automatically, without user intervention has been exhausted in the 
discussions above. 

With regard to new claim 31, Hillebrand discloses wherein said facial blemish is a 
wrinkle and wherein said facial feature is an eye within said graphical image (see 
abstract and column 6, lines 1-8). Hillebrand discloses the corner of an eye to be a 
landmark and a wrinkle to be a type of defect. 

With regard to claim 32, the discussion of claim 31 applies. 

6. Claims 4 and 13 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over the combination of Kinjo and Hillebrand and Lawton in view of U.S. 
Patent 5,835,616 to Lobo. 

With regard to claim 4, Kinjo discloses said image enhancer, wherein 
manipulating said portion, applies various image-processing techniques. Neither Kinjo 
nor Hillebrand expressly disclose blurring the appearance of the facial feature. Lobo 
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discloses enhancing a facial image by blurring the image (Abstract). Lobo teaches that 
the blurring filter is used to better set forth the facial features of the image (Abstract). 
Blurring is a well-known image enhancing technique. Therefore it would have been 
obvious to one of ordinary skill in the art at the time of invention to add blurring to the list 
of available facial feature enhancing techniques listed by Kinjo in order to better set 
forth the facial features of the image. 

With regard to claim 1 3, the discussion of claim 4 applies. The method used in 
the system is considered to be included in the system. 



Conclusion 

Applicant's amendment necessitated the new grounds of rejection presented in 
the Office Action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 706.07(a). 
Applicant is reminded of the extension of time policy as set forth in 37CFR 1 .136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of 
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the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Wes Tucker whose telephone number is 571-272-7427. 
The examiner can normally be reached on 9AM-5PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Jingge Wu can be reached on 571-272-7429. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 



Wes Tucker 



